I. Introduction: Contextualizing the King County House Price Prediction Model

Problem Statement and Objective

In the vibrant and diverse King County real estate market, including Seattle’s dynamic environment, property prices are shaped by an array of variables. The primary challenge is to construct a predictive model that can accurately estimate house prices within this area. Utilizing a comprehensive dataset that encompasses diverse house attributes, this model aims to decode the complex mechanisms influencing house pricing.

Purpose of the Model

Predictive Accuracy

The model strives to offer precise price predictions for properties in King County by effectively correlating various house features with their market prices. This aspect is crucial in understanding and quantifying how different characteristics impact the value of a property.

Analytical Insight

A key goal of the model is to unearth and interpret the multitude of factors that play a significant role in determining house prices within the region. This venture goes beyond mere statistical analysis to provide practical, real-world insights, thereby enriching the understanding of real estate dynamics for all stakeholders.

Decision Support

The model is designed to be a powerful asset for a range of users, including real estate agents, prospective buyers, and sellers. By offering accurate price predictions and deep market insights, it aids in making informed and strategic decisions in the property market.

Scope and Methodology

Data Preprocessing and Exploration

Initial data preparation is vital to ensure accuracy in the model. This stage involves cleansing the data, converting data types, and creating dummy variables for categorical features. Following this, an exploratory data analysis (EDA) is conducted to delve into the dataset’s characteristics, examining statistical summaries and relationships between variables.

Feature Selection and Model Assumptions

The process involves using statistical techniques like stepwise regression for feature selection and conducting tests like the Variable Inflation Factor (VIF) and Anderson-Darling to check for multicollinearity and normality, respectively. Additionally, diagnostic plots are used for detecting outliers.

Model Development and Validation

A range of models are employed and assessed:

Linear Models: Including Ordinary Least Squares (OLS) and Weighted Least Squares (WLS).

Regularization Techniques: Such as Ridge, Lasso, and Elastic Net to handle multicollinearity.

Robust Regression: Utilizing Huber’s method to minimize the influence of outliers.

Advanced Models: Exploring alternatives like regression trees, neural networks (NN), or support vector machines (SVM).

Model Performance Evaluation

The model’s effectiveness is evaluated using metrics like RMSE and R-squared, across both the training (70%) and testing (30%) data sets, to ensure its reliability and applicability in real-world scenarios.

Conclusion

This introduction sets the stage for a comprehensive analysis, highlighting the multifaceted approach adopted in this project. From meticulous data preparation to sophisticated modeling, the endeavor is not just to predict house prices accurately but also to provide valuable insights into King County’s real estate market.

II. Description of the Data and Quality

Dataset Overview and Detailed Description

The King County house sales dataset is a comprehensive collection of 21,613 observations, each representing a unique house sale. The dataset encompasses a variety of features that describe different aspects of the houses sold. Below is a detailed description of each variable in the dataset:

Variable Description
id Unique ID for each home sold (not used as a predictor)
date Date of the home sale
price Price of each home sold
bedrooms Number of bedrooms
bathrooms Number of bathrooms, “.5” accounts for a bathroom with a toilet but no shower
sqft_living Square footage of the apartment interior living space
sqft_lot Square footage of the land space
floors Number of floors
waterfront A dummy variable for whether the apartment was overlooking the waterfront or not
view An index from 0 to 4 of how good the view of the property was
condition An index from 1 to 5 on the condition of the apartment
grade An index from 1 to 13 about building construction and design quality
sqft_above The square footage of the interior housing space above ground level
sqft_basement The square footage of the interior housing space below ground level
yr_built The year the house was initially built
yr_renovated The year of the house’s last renovation
zipcode The zipcode area the house is in
lat Latitude coordinate
long Longitude coordinate
sqft_living15 The square footage of interior housing living space for the nearest 15 neighbors
sqft_lot15 The square footage of the land lots of the nearest 15 neighbors

Data Quality and Transformation

Data Cleaning and Transformation

The dataset’s preparation involved meticulous cleaning and transformation processes to optimize it for accurate predictive analysis. Key steps undertaken include:

  1. Exclusion of Non-Predictive Variables:
    • The id variable, representing a unique identifier for each house sale, does not contribute to predicting house prices and was therefore removed. This step is crucial in focusing the model on variables that influence the outcome (price).
    • Unlike other non-predictive variables, lat (latitude) and long (longitude) were initially retained for their crucial role in calculating geographical distances, which could potentially influence house prices.
  2. Transformation of Data Types:
    • The date variable, initially in a string format, was transformed into a numeric format. This conversion is essential for incorporating the date into statistical models, as numeric representations are more amenable to various types of analysis.
    • For variables like price, sqft_living, sqft_lot, etc., necessary conversions were performed to ensure they are in a suitable numeric format.
  3. Creation of Dummy Variables for Categorical Data:
    • Categorical variables like waterfront, view, condition, and grade were transformed into dummy variables. This transformation is pivotal for regression analysis as it allows these non-numeric variables to be effectively included in the model.
    • The process involved converting these categorical variables into a series of binary variables (0 or 1). This is particularly important for variables like waterfront, which is a binary indicator itself, and for ordinal variables like view and condition, which have intrinsic order but need to be numerically represented for modeling.
  4. Handling Special Cases in Variables:
    • For variables like bathrooms, where values like “0.5” represent bathrooms with a toilet but no shower, the data was kept as is, considering these nuances convey important information about the house’s characteristics.
  5. Grouping and Clustering of Variables:
    • The zipcode variable was transformed by extracting the first three digits, which helps in reducing the number of dummy variables and preventing the model from becoming overly complex while still capturing the geographical influences on house prices.
    • The grade variable was clustered into broader categories to simplify the model and focus on significant differences in construction and design quality.
  6. Haversine Distance Calculation:
    • To incorporate the influence of location more precisely, the Haversine distance was calculated. This involved creating a function to calculate the distance between two geographical points (latitude and longitude) and applying this to our dataset.
    • The calculation of haversine_distance is particularly significant for understanding the spatial relationships and proximity to key locations that might affect house prices.
  7. Calculation of Convergence Point:
    • The dataset was used to identify a ‘convergence point’ – a central point derived from houses with the highest values. This point served as a reference to calculate each property’s distance from a high-value central location, possibly a marker of a desirable area.
    • This step was critical in ensuring that the model accounts for locational desirability without causing data leakage, as it was based solely on the training set.
# Data Preprocessing and Transformation
set.seed(123)  # Setting a seed for reproducibility
split_index <- sample(1:nrow(df), size = 0.7 * nrow(df))
train_df <- df[split_index, ]
test_df <- df[-split_index, ]

# Remove non-numeric characters from the 'price' column and convert it to numeric
train_df$price <- as.numeric(str_replace_all(train_df$price, "[^0-9.]", ""))
test_df$price <- as.numeric(str_replace_all(test_df$price, "[^0-9.]", ""))

# Calculation of Convergence Point: Determine the convergence point for high-value homes
high_value_threshold <- quantile(train_df$price, probs = 0.999, na.rm = TRUE)  # Calculate the high-value threshold
high_value_homes <- train_df[train_df$price >= high_value_threshold, ]  # Select high-value homes
convergence_point <- c(mean(high_value_homes$lat, na.rm = TRUE), mean(high_value_homes$long, na.rm = TRUE))  # Calculate the convergence point

# Data Transformation Function
transform_data <- function(df, convergence_point) {
  # Date Transformation: Convert the 'date' column to a Date object
  df$date <- as.Date(substr(as.character(df$date), 1, 8), format="%Y%m%d")

  # Date-Time Feature Engineering: Extract various date-related features
  df$year_sold <- lubridate::year(df$date)  # Extract the year of sale
  df$month_sold <- lubridate::month(df$date)  # Extract the month of sale
  df$day_sold <- lubridate::day(df$date)  # Extract the day of sale
  df$season <- factor(lubridate::quarter(df$date), labels = c("Winter", "Spring", "Summer", "Fall"))  # Determine the season of sale
  df$week_of_year <- lubridate::week(df$date)  # Extract the week of the year of sale
  df$day_of_year <- lubridate::yday(df$date)  # Extract the day of the year of sale

  # Creating Dummy Variables: Convert categorical variables into dummy variables
  df$waterfront <- as.factor(df$waterfront)
  df$view <- as.factor(df$view)
  df$condition <- as.factor(df$condition)
  df$grade <- as.character(df$grade)
  df$grade <- ifelse(df$grade %in% c("1", "2", "3"), "1_3", df$grade)
  df$grade <- ifelse(df$grade %in% c("4", "5", "6"), "4_6", df$grade)
  df$grade <- ifelse(df$grade %in% c("8", "9", "10"), "8_10", df$grade)
  df$grade <- ifelse(df$grade %in% c("11", "12", "13"), "11_13", df$grade)
  df <- dummy_cols(df, select_columns = c('zipcode', 'view', 'condition', 'grade'))

  # Haversine Distance Function: Calculate the distance between two points on Earth's surface
  haversine_distance <- function(lat1, long1, lat2, long2) {
    R <- 6371  # Earth radius in kilometers
    delta_lat <- (lat2 - lat1) * pi / 180
    delta_long <- (long2 - long1) * pi / 180
    a <- sin(delta_lat/2)^2 + cos(lat1 * pi / 180) * cos(lat2 * pi / 180) * sin(delta_long/2)^2
    c <- 2 * atan2(sqrt(a), sqrt(1 - a))
    d <- R * c  # Calculate the haversine distance
    return(d)
  }

  # Applying Haversine Distance to the Dataset: Calculate the distance to the convergence point
  df$distance_to_convergence <- mapply(haversine_distance, df$lat, df$long,
                                       MoreArgs = list(lat2 = convergence_point[1], long2 = convergence_point[2]))

  # Removing no longer needed columns
  df <- df[, !(names(df) %in% c("id", "lat", "long", "date", "zipcode", "view", "condition", "grade"))]

  return(df)
}

# Applying the transformation function to training and test sets
train_df <- transform_data(train_df, convergence_point)  # Transform the training data
test_df <- transform_data(test_df, convergence_point)    # Transform the test data

Training Data Header

price bedrooms bathrooms sqft_living sqft_lot floors waterfront sqft_above sqft_basement yr_built yr_renovated sqft_living15 sqft_lot15 year_sold month_sold day_sold season week_of_year day_of_year zipcode_98001 zipcode_98002 zipcode_98003 zipcode_98004 zipcode_98005 zipcode_98006 zipcode_98007 zipcode_98008 zipcode_98010 zipcode_98011 zipcode_98014 zipcode_98019 zipcode_98022 zipcode_98023 zipcode_98024 zipcode_98027 zipcode_98028 zipcode_98029 zipcode_98030 zipcode_98031 zipcode_98032 zipcode_98033 zipcode_98034 zipcode_98038 zipcode_98039 zipcode_98040 zipcode_98042 zipcode_98045 zipcode_98052 zipcode_98053 zipcode_98055 zipcode_98056 zipcode_98058 zipcode_98059 zipcode_98065 zipcode_98070 zipcode_98072 zipcode_98074 zipcode_98075 zipcode_98077 zipcode_98092 zipcode_98102 zipcode_98103 zipcode_98105 zipcode_98106 zipcode_98107 zipcode_98108 zipcode_98109 zipcode_98112 zipcode_98115 zipcode_98116 zipcode_98117 zipcode_98118 zipcode_98119 zipcode_98122 zipcode_98125 zipcode_98126 zipcode_98133 zipcode_98136 zipcode_98144 zipcode_98146 zipcode_98148 zipcode_98155 zipcode_98166 zipcode_98168 zipcode_98177 zipcode_98178 zipcode_98188 zipcode_98198 zipcode_98199 view_0 view_1 view_2 view_3 view_4 condition_1 condition_2 condition_3 condition_4 condition_5 grade_1_3 grade_4_6 grade_7 grade_8_10 grade_11_13 distance_to_convergence
475000 4 2.50 2040 16200 2.0 0 2040 0 1997 0 2530 15389 2015 3 7 Winter 10 66 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 24.409710
316000 4 1.50 2120 46173 2.0 0 2120 0 1974 0 2000 46173 2015 5 8 Spring 19 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 20.734826
802000 4 2.25 2130 8734 2.0 0 2130 0 1961 0 2550 8800 2014 9 4 Summer 36 247 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 9.318110
905000 4 2.50 3330 9557 2.0 0 3330 0 1995 0 3360 9755 2015 3 25 Winter 12 84 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 13.647505
700000 4 2.25 2440 9450 1.5 0 2440 0 1947 2014 1720 7503 2014 10 30 Fall 44 303 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 9.711104
178500 3 1.00 900 10511 1.0 0 900 0 1961 0 1460 10643 2015 2 12 Winter 7 43 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 38.067677

Statistical Analysis and Correlation

Exploratory Data Analysis (EDA)

The exploratory data analysis (EDA) conducted on the King County house sales dataset is an in-depth exploration aimed at uncovering patterns, anomalies, and relationships within the data. This comprehensive EDA includes a variety of analyses to gain a holistic understanding of the dataset’s characteristics.

  1. Distribution Analysis of Continuous Variables:
    • This analysis focuses on continuous variables like price, sqft_living, sqft_lot, and others. Key aspects include examining their distributions, identifying potential outliers, and understanding their range and central tendencies.
    • Histograms and box plots are used to visualize these distributions, which can reveal skewness, kurtosis, and other distributional characteristics important for model assumptions.
  2. Categorical Variable Analysis:
    • The distribution and count of categorical variables such as bedrooms, bathrooms, floors, waterfront, view, condition, and grade are analyzed.
    • Bar plots and frequency tables help in understanding the prevalence of different categories and their potential impact on house prices.
  3. Correlation Analysis:
    • Understanding how continuous variables correlate with each other and, more importantly, with the target variable price.
    • A correlation matrix and corresponding heat map provide a visual and quantitative view of these relationships, highlighting variables that might have a strong positive or negative relationship with house prices.
  4. Temporal Trends Analysis:
    • Analyzing the influence of time-related features such as year_sold, month_sold, and season on house prices.
    • Time series plots and seasonal decomposition can reveal trends, seasonality, and cyclical patterns in house prices.
  5. Geographical Influence Analysis:
    • Investigating the spatial aspect by analyzing the distance_to_convergence variable.
    • Scatter plots or spatial heat maps can illustrate if proximity to the high-value convergence point influences house prices.

Continuous Variable Analysis

This analysis focuses on continuous variables like price, sqft_living, sqft_lot, and others. Key aspects include examining their distributions, identifying potential outliers, and understanding their range and central tendencies.

Price vs. Square Footage of Living Space

In the scatter plot above, we compare the price of homes against their sqft_living (square footage of interior living space). This visualization allows us to explore the relationship between these two variables.

The histogram above displays the distribution of sqft_living. It reveals that the variable is right-skewed, with most homes having smaller living spaces and relatively fewer very large living spaces.

Price vs. Square Footage of Lot

The scatter plot above compares price against sqft_lot (square footage of land space). It helps us understand if there’s any relationship between the size of the lot and the sale price.

The histogram above visualizes the distribution of sqft_lot. Similar to sqft_living, this variable is right-skewed, with most homes having smaller lot sizes and relatively fewer very large lots.

Price vs. Square Footage Above Ground

In the scatter plot above, we compare price against sqft_above (square footage of the interior housing space above ground level). This analysis helps us explore the impact of above-ground living space on home prices.

The histogram above shows the distribution of sqft_above. It suggests that most homes have similar above-ground square footage, with relatively fewer having significantly larger or smaller above-ground spaces.

Price vs. Square Footage of Basement

Excluding homes that do not have a basement.

The scatter plot above compares price against sqft_basement (square footage of the interior housing space below ground level). This visualization helps us understand if the presence and size of a basement influence home prices.

The histogram above visualizes the distribution of sqft_basement. It indicates that most homes have little to no basement space, while some have larger basement areas.

Price vs. Year Built

The scatter plot above compares price against the year when homes were initially built (yr_built). This analysis helps us understand how the age of a home relates to its sale price.

The histogram above displays the distribution of yr_built. It provides insights into the distribution of home ages in the dataset.

Price vs. Year of Last Renovation

Excluding homes that did not have a documented renovation.

In the scatter plot above, we compare price against the year of the last renovation (yr_renovated). This analysis helps us understand whether recent renovations impact home prices.

The histogram above visualizes the distribution of yr_renovated. It provides insights into the distribution of renovation years in the dataset.

Price vs. Distance to Convergence

The scatter plot above compares price against distance_to_convergence. This analysis helps us explore whether the distance to a convergence point impacts home prices.

Categorical Variable Analysis

The distribution and count of categorical variables such as bedrooms, bathrooms, floors, waterfront, view, condition, and grade are analyzed.

Price vs. Bedrooms

The scatter plot above compares price against the number of bedrooms. This visualization helps us understand how the number of bedrooms influences home prices.

The bar plot above displays the distribution of the bedrooms variable, showing the frequency of each bedroom count.

Price vs. Bathrooms

In the scatter plot above, we compare price against the number of bathrooms. This analysis helps us explore the relationship between the number of bathrooms and home prices.

The bar plot above visualizes the distribution of the bathrooms variable, showing the frequency of each bathroom count.

Price vs. Floors

The scatter plot above compares price against the number of floors. This analysis helps us understand how the number of floors in a home relates to its sale price.

The bar plot above displays the distribution of the floors variable, showing the frequency of each floor count.

Price vs. Waterfront

In the scatter plot above, we compare price against the waterfront variable. This visualization helps us explore how having a waterfront view impacts home prices.

The bar plot above visualizes the distribution of the waterfront variable, showing the frequency of waterfront and non-waterfront properties.

Price vs. View

The scatter plot above compares price against the view variable, which represents the quality of the property’s view. This analysis helps us explore how the view quality impacts home prices.

The bar plot above displays the distribution of the view variable, showing the frequency of different view quality ratings.

Price vs. Condition

In the scatter plot above, we compare price against the condition variable, which represents the condition of the property. This analysis helps us explore how property condition relates to home prices.

The bar plot above visualizes the distribution of the condition variable, showing the frequency of different condition ratings.

Price vs. Grade

The scatter plot above compares price against the grade variable, which has been aggregated into categories as per the provided header. This analysis helps us explore how the grade of construction and design impacts home prices.

The bar plot above displays the distribution of the grade_category variable, showing the frequency of different grade categories.

Correlation Analysis

Understanding how continuous variables correlate with each other and, more importantly, with the target variable price.

Top 20 Correlation Values with Price
Variable Correlation with Price
price 1.0000000
sqft_living 0.7055923
sqft_above 0.6090981
sqft_living15 0.5872083
bathrooms 0.5337709
grade_category 0.5300297
grade_aggregate 0.5300297
grade_11_13 0.4909842
view_category 0.4016872
sqft_basement 0.3278610
bedrooms 0.3141109
view_4 0.3107036
grade_8_10 0.3103275
zipcode_98004 0.2685431
floors 0.2593524
zipcode_98039 0.2087856
zipcode_98040 0.1916889
view_3 0.1771779
zipcode_98112 0.1717757
view_2 0.1570317

Correlation Graphics Analysis

In the table above, we’ve displayed the top 20 correlation values with the target variable price, sorted by their absolute values. Here are some of the key findings:

  1. Positive Correlations with Price:
    • Variables such as sqft_living, sqft_above, sqft_living15, and bathrooms exhibit strong positive correlations with the target variable. This suggests that as these variables increase, the house price tends to increase as well.
    • Features like grade_11_13, view_4, and grade_8_10 also show positive correlations, indicating that higher-grade properties and better views tend to have higher prices.
  2. Negative Correlations with Price:
    • No negative correlations are present in the top 20. This means that none of the examined features strongly suggest a decrease in price as they increase.
  3. Feature Importance:
    • The strength of the correlations helps us understand the importance of these variables in predicting house prices. Features like sqft_living and grade_11_13 appear to be strong predictors of price.
    • Variables related to location, such as zipcode_98004, zipcode_98039, and zipcode_98040, also have notable positive correlations, indicating the significance of location in price determination.

Geographical Influence Analysis

Investigating the spatial aspect by analyzing the distance_to_convergence variable.

# Graphical Analysis Code

Conclusion

This detailed review of the King County house sales dataset underscores the thorough preparation undertaken for the predictive analysis. The dataset’s diverse variables, both continuous and categorical, have been meticulously processed and analyzed, providing a robust foundation for developing the predictive model. With the comprehensive EDA and graphical analysis, we gain valuable insights into the correlations and distributions within the data, setting the stage for effective model building and accurate house price prediction.